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DETAILED ACTION 

Response to Amendment 

1 . The Examiner acknowledges the amended claims filed on February 1 1 , 2009. 
Claims 1, 21 and 54 have been amended. Claims 6, 8, 9, 26, 27, 39, 41, 43, 48-50, 55 
and 57-60 have been cancelled. 

Response to Arguments 

2. Applicant's arguments with respect to claims 1 and 21 have been considered but 
are moot in view of the new grounds of rejection based on previously presented prior 
art. 

3. Applicant's arguments with respect to claim 54 have been considered but are 
moot in view of the new grounds of rejection. 

4. Applicant's arguments filed February 1 1 , 2009 have been fully considered but 

they are not persuasive. The Applicant argues the following: 

a. Takahashi fails to teach, disclose or suggest at least "saliencv circuitry for 
automatically generating an image related second saliencv signal in response to 
the image signal" 

The Office Action contends that Takahashi teaches this as follows: 

saliency circuitry for automatically generating an image related 
second saliency signal in response to the image signal (Takahashi 
further teaches that the auxiliary information includes a degree of 
importance of said persons appearing in the image data, and that 
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said degree of importance can be determined based on the time 
length of a scene where a particular person set by the user appear; 
see page 8^01 19.) 
(Office Action, p. 8-9, emphasis added.) 

Applicant respectfully disagrees. In Takahashi, the user inputs the 
degree of importance: the user creates tags or button labels in advance 
(e.g., "son", "daughter", "exciting" etc.) and then the user enters a value 
for these labels while the scene is being captured. Although this point is 
not explicit with regard to person-in-the-scene tags, Takahashi makes this 
clear when discussing a "degree of importance" value for the "exciting" 
tag: 

The simplest method of setting the value of exciting is setting the 
value of importance at either "0" (=not important) or "1" (=most 
important). However, one of the value buttons shown in the menu 
on the monitor or the menu in the viewfinder may be selected by 
the menu method, i.e., in the same manner as that described for 
the case of answering to a question as to whether the type of 
auxiliary information should be selected or not. At this time, 
values in increments of "0.1" may be inputted between "0" and 
"1" by combination of pressing the power key and the 
recording key, and one of these values may be selected. 
Alternatively, when an exciting button is provided at the upper 
surface of the body of the combined camera and VTR, the user 
may input a value of exciting by operating this button. 
Furthermore, a value of exciting may be inputted by sensing the 
degree of exciting of the user from the fingers of the user which are 
put on a pressure sensor or a sweat sensor provided at the upper 
surface of the body. 

(Takahashi, para. 0119, emphasis added.) 

Thus, even assuming (for the sake of argument) that the degree of 
importance of a person in the scene is an "image related second saliency signal", 
Takahashi does not teach "automatically generating" this signal. 
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> The Examiner disagrees with the Applicant. However, that Examiner 
would like to point out that Takahashi teaches that the saliency signal can be 
ether input by the user as well as automatically generated by the camera upon 
detection of excitement of the user using either a pressure sensor or a sweat 
sensor provided in the camera so that the camera would automatically assign a 
"degree of importance" based on the detected excitement of the user as also 
discussed in If 0119. Therefore, the Examiner understands that Takahashi 
discloses "saliency circuitry for automatically generating an image related second 
saliency signal in response to the image signal" as claimed since the "degree of 
interest" is automatically generated upon measuring the excitement of the user 
while capturing the image signal. 



b. Takahashi fails to teach, disclose or suggest at least "circuitry for 
combining said saliency signals to provide a complex saliency signal" 

The Office Action contends that Takahashi teaches this as follows: 

circuitry for combining said saliency signals to form a complex 
saliency signal (Takahashi discloses that the auxiliary information 
has persons information and degree of importance of said persons 
and that the auxiliary information is stored with the image signal 
and further discloses that the camera would also display the 
auxiliary information on a display screen (As shown in figs. 23, the 
user would set the auxiliary information and the set auxiliary 
information would be displayed in the display as shown in figs. 25 
and 26 (see persons information and the degree of importance 
displayed on the display); page 87 I] 0117 - page 9, I] 0124)) (This 
teaches the combination of the two saliency signals (the 
persons information and the degree of importance information 
stored/displayed together in association with the image) to 
create a complex saliency signal as claimed .) 
(Office Action, p. 8-9, emphasis added.) 
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Applicant respectfully disagrees with the contention. Applicant assumes (for the 
sake of argument) that the person-of-interest information and the degree-of- 
importance information are each a "saliency signal". Even so, Takahashi does 
not teach or suggest "combining" these two pieces of information. Takahashi 
merely teaches storing these two pieces of information, which Applicant 
respectfully submits is not the same as "combining" as recited in claim 44. 
Furthermore, even assuming (for the sake of argument) that storing these two 
pieces of information is the same as combining them, Applicant respectfully that 
the stored information is not "complex" as recited in claim 44. 
> The Examiner disagrees with the Applicant. As discussed in the previous 
Office Action, Takahashi discloses that the saliency signals (wherein the 
Examiner is interpreting the person information and the degree of importance as 
saliency signals) are displayed together (See figs. 20(a) and 26(b)). Therefore, 
by at least teaching that the person information and the degree of importance are 
displayed together in the display of the camera, Takahashi teaches combining 
the signals to create a complex signal that is used for creating a display signal to 
inform the user about the person information and the degree of importance in the 
video being displayed. Therefore, the Examiner understands that Takahashi 
discloses "circuitry for combining said saliency signals to provide a complex 
saliency signal" as claimed. 
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c. Claim 54 is amended to recite "a picture taking control for selectively 

activating the electronic camera to store an image to a memory and... picture 

selection circuitry for selectively passing the image to the memory in response to 

the saliency signal" (emphasis added). Applicant respectfully submits that 

Metcalfe fails to teach, disclose or suggest at least this feature. 

Metcalfe discloses a video camera 101 with a level of interest (LOI) button 112, 

through which a user inputs his subject level of interest in content as it records. 

(p. 4, lines 25-30.) The LOI is stored along with the content, (p. 5, lines 5-10.) 

Metcalfe teaches that the LOI information is used during playback: 

The LOI information can be used in a number of ways during 
playback of the video frames. In one embodiment, the speed of 
playback is a function of the LOI signal from the frame currently 
being replayed, 
(p. 5 lines 15-20.) 

As shown in Fig. 3, the use can select an LOI threshold, whereby a 
replay controller controls the replay apparatus such that only video 
frames having an LOI above the selected threshold are replayed., 
(p. 5 lines 15-20.) 

The LOI signals associated with the video frames can also be used 
to automatically produce a printout of relatively interesting parts of 
the tape. (p. 5 lines 15-20.) 

But even assuming (for the sake of argument) that the LOI information is a 

saliency signal, the LOI information is not used to control the storing of images. 

In contrast, in the embodiment defined by amended claim 54 the saliency signal 

affects the storage of images: "a picture taking 
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control for selectively activating the electronic camera to store an image to a 
memory .... and picture selection circuitry for selectively passing the image to the 
memory in response to the saliency signal" 

> The Examiner acknowledges that the above mentioned newly presented 
limitations are not taught in the Metcalfe reference. However, a new ground of 
rejection has been made in view of a teaching found in the Takahashi reference, 
which is now presented to modify the teaching of Metcalfe to show the limitations 
as discussed in the rejections under 35 USC 103 as discussed in this Office 
Action. 

d. Applicant assumes (for the sake of argument) that the level of interest 
(LOI) signal in Metcalfe corresponds to a saliency signal. Metcalfe teaches that 
the LOI can have a range of values (p. 4 lines 30-32) or can be essentially 
continuous (p. 4 line 33 to p. 5 line 3). However, Metcalfe does not disclose 
"providing a maximum value" for the LOI signal, and certainly not that this 
maximum value is provided "contemporaneously with activation of the picture 
taking control". 

Applicant further assumes (for the sake of argument) that the auxiliary 
information in Takahashi corresponds to a saliency signal. Takahashi teaches 
that the type of auxiliary information and the value of this information are both 
input by a user. (See Fig. 23(a) and para. 01 19.) However, Takahashi does not 
disclose "providing a maximum value" for the auxiliary information and certainly 
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not that this maximum value is provided "contemporaneously with activation of 
the picture taking control". 

> The Examiner disagrees. The Examiner noted that Metcalfe further 
discloses that the record button 112 and the button 111 can be combined into a 
single combination key, and depressing the combination key starts the recording 
of video while subsequent pressure applied by the user generates a LOI signal 
for recording with the captured video frames (See page 6, lines 6-12). It is noted 
that the word "contemporaneous" is defined as "Originating, existing, or 
happening during the same period of time" (Definition from "The American 
Heritage® Dictionary of the English Language", Fourth Edition, 2000). Therefore, 
by teaching that the combination button is initially depress for activation while a 
subsequent pressure applied by the user that in response the camera would 
assign a particular LOI to the video frames being recorded, Metcalfe discloses "a 
circuit for providing a maximum value for the saliency signal contemporaneously 
with activation of the picture taking button" as claimed since even if the LOI is not 
assign at the same time, the LOI is assign at a time that can be considered to be 
occurring during a particular period of time. Therefore, the Examiner 
understands that the combined teaching of Metcalfe in view of Takahashi as 
discussed and analyzed in claims 1 and 21 teaches all the limitations as 
presented. 
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e. Applicant respectfully submits that claims 1 , 3-5, 1 0-21 , 23-25, 28-38, 40, 

and 42 are allowable for at least the reason that the proposed combination of 

Metcalfe in view of Takahashi does not disclose, teach, or suggest at least the 

feature of "saliency circuitry for combining said first and second saliency signals 

to form a complex saliency signal". The Office Action acknowledges that Metcalfe 

does not explicitly disclose this feature, but contends that Takahashi teaches the 

feature as follows: 

circuitry for combining said saliency signals to form a complex 
saliency signal (Takahashi discloses that the auxiliary information 
has persons information and degree of importance of said persons 
and that the auxiliary information is stored with the image signal 
and further discloses that the camera would also display the 
auxiliary information on a display screen (As shown in figs. 23, the 
user would set the auxiliary information and the set auxiliary 
information would be displayed in the display as shown in figs. 25 
and 26 (see persons information and the degree of importance 
displayed on the display); page 87 l[ 0117 - page 9, l[ 0124)) {This 
teaches the combination of the two saliency signals (the 
persons information and the degree of importance information 
stored-displayed together in association with the image) to 
create a complex saliency signal as claimed .) 
(Office Action, p. 8-9, emphasis added.) 



Applicant respectfully disagrees with the contention. Applicant assumes (for the 
sake of argument) that the person-of-interest information and the degree-of- 
importance information are each a "saliency signal". Even so, Takahashi does 
not teach or suggest "combining" these two pieces of information. Takahashi 
merely teaches storing these two pieces of information, which Applicant 
respectfully submits is not the same as "combining" as recited in claim 40. 
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> The Examiner disagrees with the Applicant. As discussed in the previous 
Office Action, Takahashi discloses that the saliency signals (wherein the 
Examiner is interpreting the person information and the degree of importance as 
saliency signals) are displayed together (See figs. 20(a) and 26(b)). Therefore, 
by at least teaching that the person information and the degree of importance are 
displayed together in the display of the camera, Takahashi teaches combining 
the signals to create a complex signal that is used for creating a display signal to 
inform the user about the person information and the degree of importance in the 
video being displayed. Therefore, the Examiner understands that Takahashi 
discloses "circuitry for combining said saliency signals to provide a complex 
saliency signal" as claimed. 

f. Applicant respectfully submits that claims 51 -53 are allowable for at least 
the reason that Takahashi does not disclose, teach, or suggest at least the 
feature of "a buffer for receiving the input picture signals and having a capacity 
for the input picture signals determined in response to the value of the saliency 
signal". The Office Action contends that Takahashi teaches this feature as 
follows: 

By teaching adjusting the amount of data to be transmitted based 
on the length allowed for transmission and also based on the 
priority of the video signals to be transmitted, Takahashi inherently 
discloses "a buffer for receiving said image signal, the buffer having 
a capacity arranged to be controlled by the value of the saliency 
signal during operation of the camera apparatus" as claimed since 
the amount being adjusted in the apparatus needs to be in a 
particular buffer or memory prior to transmission in order to properly 
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select the video signals with higher priority). Takahashi further 
discloses that by adjusting the amount of data to be transmitted 
based on the length allowed for transmission and also based on the 
priority of the video signals to be transmitted the camera may select 
as many as possible the most important video signals from the 
video data for transmission. 
(Office Action, p. 44.) 

Applicant assumes the following for the sake of argument: that Takahashi 

teaches "adjusting the amount of data to be transmitted based on the length 

allowed for transmission and also based on the priority of the video signals to be 

transmitted"; that Takahashi inherently includes a buffer to store this data; and 

that the priority of the video signals corresponds to the claimed "saliency signal". 

Even so, the conclusion drawn in the Office Action from these premises is faulty: 

the mere presence of a buffer to store data for transmission and the (assumed) 

fact that the amount of data transmitted is adjusted does not necessarily lead to 

the conclusion that the capacity of the buffer is controlled by the value of the 

saliency signal. It is also possible that the amount of data stored in the buffer is 

completely unrelated to the saliency . and only the amount of data transmitted is 

controlled by the saliency. According to well-established Federal case law, 

"Anticipation by inherent disclosure is appropriate only when the reference 

discloses prior art that must necessarily include the unstated limitation." Atofina 

v. Great Lakes Chemical Corp., 441 F.3d 991, 1000 (Fed. Cir. 2006). Since 

Applicant has demonstrated that Takahashi does not necessarily include the 

claimed feature, the finding of inherency is inadequate in showing why "a 

buffer... having a capacity for the input picture signals determined in response to 
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the value of the saliency signal" is "necessarily present" in the reference. 
Consequently, the conclusion drawn in the Office Action not adequately 
supported, and the rejection of claim 51 should be withdrawn. 
> The Examiner notes that it is possible that an amount of data stored in a 
buffer is completely unrelated to the saliency as argued by the Applicant. 
However, As discussed and analyzed in the previous Office Action, in the 
embodiment of Takahashi used to teach the limitations in question, Takahashi 
teaches a transmission process in which the amount of data to be transmitted is 
determined based on the cost of transmission service, thus wherein the allowed 
length of data (L) is determined and compared to the amount of data to be 
transmitted, and if the amount of data to be transmitted exceeds the determined 
amount of data allowed for transmission (See fig. 47) the apparatus would select 
video section with a priority higher than a threshold value. If after selecting those 
video with the priority higher than a threshold, the length of data still more than 
the allowed length, the apparatus would increase the priority threshold, however, 
if the length is less that the allowed length, the video signals are transmitted 
(Page 12, ^ 0151-0154). It is noted by the Examiner by controlling the length of 
data to be transmitted, an amount of data need to be measured in memory in 
order to be compared, this teaches the use of a buffer of a memory section 
working as a buffer that is required to perform said control of data length and 
comparison of the selected data with said data length, which could be a section 
of the main memory or a separate memory, since that separate memory (buffer) 
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or section in the memory includes a particular amount of data from the selected 
data to be measured for comparison with the allowed length of data for 
transmission. Furthermore, by teaching controlling said amount of data (which is 
determined based on the priority of data in response to the allowed length of data 
for transmission) to be transmitted based on the allowed length of data for 
transmission (which is determined on cost of transmission), Takahashi discloses 
the use of a buffer for receiving the input picture signals and having a capacity for 
the input picture signals determined in response to the value of the saliency 
signal since based on the importance of the data, the capacity of a section of the 
buffer (section of memory or separate memory as discussed above) for 
transmission is controlled. Therefore, the Examiner understands that the 
Takahashi reference teaches the limitations as presented. 

g. Applicant notes that the Office Action specifically relies (p. 44) on an 
alleged finding of inherency for a teaching of the claimed feature noted above. 
However, Applicant also notes that the rejection is an obviousness rejection 
under §103 rather than an anticipation rejection under §103. It is therefore 
unclear to Applicant whether the rejection of claim 51 also relies on the 
knowledge of a person of ordinary skill in the art (see p. 1 0) for teaching "a 
buffer.. .having a capacity for the input picture signals determined in response to 
the value of the saliency signal". To the extent that the rejection relies on the 
knowledge of a person of ordinary skill in the art instead of inherency for teaching 
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this claimed feature, Applicant respectfully submits that this claimed feature is not 
something within such a person's knowledge. Accordingly, the proposed 
combination of Takahashi in view of the knowledge of a person of ordinary skill in 
the art does not teach at least the claimed feature noted above that is recited in 
claim 51 . Therefore, a prima facie case establishing an obviousness rejection 
has not been made, and the rejection should be withdrawn. 
> The Examiner would like to point out that the rejection of claim 51 has 
been made as an obviousness rejection under §103. Note that the Examiner is 
using two different embodiments from the Takahashi reference (first and third 
embodiments as shown in the Takahashi references). The teaching of a buffer 
for receiving the input picture signals and having a capacity for the input picture 
signals determined in response to the value of the saliency signal has been 
addressed as being inherent in one of the embodiments of the Takahashi 
reference (third embodiment) as required to perform said control of data amount 
to be transmitted since a buffer, memory or section in main memory is required to 
perform the selection of the image data for comparison to adjust the amount of 
data for transmission based on the priority of the image signal. Thus, the 
teaching of "a buffer for receiving the input picture signals and having a capacity 
for the input picture signals determined in response to the value of the saliency 
signal" has not been taught relying with the use of Official Notice or on the 
knowledge of a person of ordinary skill in the art. The limitations of claim 51 
have been rejected as being obvious over one of the embodiments (third) of 
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Takahashi in combination with a different embodiment (first) of Takahashi that 
does not teach such limitations. The Examiner understands that modifying the 
first embodiment of Takahashi with the teaching of the third embodiment of 
Takahashi would provide the benefit of allowing selection of as many as possible 
the most important video signals from the video data for transmission as 
suggested by Takahashi (Page 12, 0154) (As discussed in the previous Office 
Action). Therefore, the Examiner understands that the combined teaching of the 
first and third embodiments in Takahashi teaches the limitations of claim 51 as 
presented. 

Claim Rejections - 35 USC § 102 

5. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 1 02 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public 
use or on sale in this country, more than one year prior to the date of application for patent in the United 
States. 

6. Claims 44-47 are rejected under 35 U.S.C. 102(b) as being anticipated by 
Takahashi, US 2002/0041757 A1. 

Regarding claim 44, Takahashi discloses an imaging system comprising an 
electronic camera (See figs. 4(a), 4(b), 8, 9(a), 9(b), 1 0, and 1 1 ) for producing an image 
signal, a physically or mechanically operable user control (Takahashi discloses the use 
of buttons 109 to set auxiliary information (which the Examiner is interpreting as the 
saliency information) related to the image data being captured; page 6, H 0106. 
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Takahashi further discloses the use of a pressure sensor 109a and sweat sensor 109b 
to determine the auxiliary information related to the image data (page 6, H 01 12 - page 
7, U 01 14) for receiving an input from a user and for generating a first saliency signal 
while the image signal is being produced (Takahashi further discloses that the auxiliary 
information includes information related to the persons (i.e. son, daughter, friend, father, 
mother) (see figs. 23(a) and 23 (b); page 8, 1f 01 1 7 - page 9, U 01 24)) for receiving an 
input from a user and for generating, in response to the input from the user a saliency 
signal (auxiliary information having information such as persons information)), saliency 
circuitry for automatically generating an image related second saliency signal in 
response to the image signal (Takahashi further teaches that the auxiliary information 
includes a degree of importance of said persons appearing in the image data, and that 
said degree of importance can be determined based on the time length of a scene 
where a particular person set by the user appear; see page 8, If 01 19.), and circuitry for 
combining said saliency signals to form a complex saliency signal (Takahashi discloses 
that the auxiliary information has persons information and degree of importance of said 
persons and that the auxiliary information is stored with the image signal and further 
discloses that the camera would also display the auxiliary information on a display 
screen (As shown in figs. 23, the user would set the auxiliary information and the set 
auxiliary information would be displayed in the display as shown in figs. 25 and 26 (see 
persons information and the degree of importance displayed on the display); page 8, U 
01 1 8 - page 9, H 01 24)) (This teaches the combination of the two saliency signals (the 
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persons information and the degree of importance information stored/displayed together 
in association with the image) to create a complex saliency signal as claimed.). 

Regarding claim 45, Takahashi discloses that the operation of at least a part of 
the electronic camera is arranged to be controlled in response to the complex saliency 
signal (Takahashi discloses that the auxiliary information has persons information and 
degree of importance of said persons and that the auxiliary information is stored with 
the image signal and further discloses that the camera would also display the auxiliary 
information on a display screen (As shown in figs. 23, the user would set the auxiliary 
information and the set auxiliary information would be displayed in the display as shown 
in figs. 25 and 26 (see persons information and the degree of importance displayed on 
the display); page 8, U 01 18 - page 9, 1f 0124)) being arranged to be controlled in 
response to the saliency signal (based on the auxiliary information the display would 
display the auxiliary information as shown in figs. 25 and 26). 

Regarding claim 46, Takahashi discloses a separate user operable picture 
taking control for permitting enabling the electronic camera to take pictures (shooting 
button 104 as shown in fig. 4 (b)). 

Regarding claim 47, Takahashi discloses that the first saliency signal can have 
more than two values (As shown in figs. 23 (a) and 23 (b), Takahashi discloses that 
more than one person can be selected (i.e. the son and the daughter); page 8, U 01 19). 
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Claim Rejections - 35 USC § 103 

7. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

8. Claims 1, 3-5, 10-21, 23-25, 28-38, 40, 42, 54 and 56 are rejected under 35 
U.S.C. 103(a) as being unpatentable over Metcalfe, AU 743216 B in view of 
Takahashi, US 2002/0041757 A1. 

Regarding claim 1 , claim 1 recites "... a saliency signal that (a) can change in 
value between at least three different discrete values while the image signal is being 
produced, or (b) can have values that are continuously variable while the image signal 
is being produced". It is noted by the Examiner that the term " can " is non-limiting and 
therefore has not been given patentable weight during examination of the claims on 
their merits. Language that suggests or makes optional but does not require steps to be 
performed or does not limit a claim to a particular structure does not limit the scope of a 
claim or claim limitation. MPEP §2106. 

The subject matter of a properly construed claim is defined by the terms that limit its 
scope. It is this subject matter that must be examined. As a general matter, the 
grammar and intended meaning of terms used in a claim will dictate whether the 
language limits the claim scope. Language that suggests or makes optional but does 
not require steps to be performed or does not limit a claim to a particular structure does 
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not limit the scope of a claim or claim limitation. The following are examples of language 
that may raise a question as to the limiting effect of the language in a claim: 

(A) statements of intended use or field of use, 

(B) "adapted to" or "adapted for" clauses, 

(C) "wherein" clauses, or 

(D) "whereby" clauses. 

This list of examples is not intended to be exhaustive. See also MPEP § 21 1 1 .04. 
USPTO personnel are to give claims their broadest reasonable interpretation in light of 
the supporting disclosure. In re Morris, 127 F.3d 1048, 1054-55, 44 USPQ2d 1023, 
1027-28 (Fed. Cir. 1997). Limitations appearing in the specification but not recited in the 
claim should not be read into the claim. E-Pass Techs., Inc. v. 3Com Corp., 343 F.3d 
1364, 1369, 67 USPQ2d 1947, 1950 (Fed. Cir. 2003) (claims must be interpreted "in 
view of the specification" without importing limitations from the specification into the 
claims unnecessarily). In re Prater, 415 F.2d 1393, 1404-05, 162 USPQ 541, 550- 551 
(CCPA 1969). See also In re Zletz, 893 F.2d 319, 321-22, 13 USPQ2d 1320, 1322 
(Fed. Cir. 1989) ("During patent examination the pending claims must be interpreted as 
broadly as their terms reasonably allow.... The reason is simply that during patent 
prosecution when claims can be amended, ambiguities should be recognized, scope 
and breadth of language explored, and clarification imposed.... An essential purpose of 
patent examination is to fashion claims that are precise, clear, correct, and 
unambiguous. Only in this way can uncertainties of claim scope be removed, as much 
as possible, during the administrative process."). 
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Metcalfe discloses a camera apparatus (See fig. 1) comprising an electronic 
camera (See fig. 1) for producing an image signal, a user operable picture taking control 
(111 as shown in fig. 1 ) for selectively activating the electronic camera to take pictures, 
and an additional physically or mechanically operable user control (1 12 as shown in fig. 
1 , note that the limitations "an additional physically or mechanically operable user 
control" are written as optional elements by using the word "or") for receiving an input 
from a user and for generating, in response to the input from the user a saliency signal 
(Metcalfe discloses generating a level of interest (LOI) set by the user when using 
button 112) that (a) can change in value between at least three different discrete values 
while the image signal is being produced (Metcalfe discloses assigning a level of 
interest to the image data being recorded, wherein the user can variably assign a 
plurality of level of interest through the capture of the a video sequence; see Fig. 3. 
This teaches generating a saliency signal that can change in value between at least 
three different discrete values while the image signal is being produced), or (b) can 
have values that are continuously variable while the image signal is being produced 
(Metcalfe discloses assigning a level of interest to the image data being recorded, 
wherein the user can variably assign a plurality of level of interest through the capture of 
the a video sequence; see Fig. 3. This teaches generating a saliency signal that can 
have values that are continuously variable while the image signal is being produced), a 
circuit for providing a maximum value for the saliency signal contemporaneously with 
activation of the picture taking button (Metcalfe further discloses that the record button 
1 1 2 and the button 111 can be combined into a single combination key, and depressing 
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the combination key starts the recording of video while subsequent pressure applied by 
the user generates a LOI signal for recording with the captured video frames (See page 
6, lines 6-12). It is noted that the word "contemporaneous" is defined as "Originating, 
existing, or happening during the same period of time" (Definition from "The American 
Heritage® Dictionary of the English Language", Fourth Edition, 2000). Therefore, by 
teaching that the combination button is initially depress for activation while a 
subsequent pressure applied by the user that in response the camera would assign a 
particular LOI to the video frames being recorded, Metcalfe discloses "a circuit for 
providing a maximum value for the saliency signal contemporaneously with activation of 
the picture taking button" as claimed since even if the LOI is not assign at the same 
time, the LOI is assign at a time that can be considered to be occurring during a 
particular period of time), and a memory (120 as shown in fig. 1) arranged for storing 
the image signal and the saliency signal (page 4, lines 5-13; page 5, lines 4-35), 
operation of at least a part of the camera apparatus while the electronic camera is 
activated to take pictures (Metcalfe discloses setting the LOI while the camera is 
activated to take pictures, and storing said LOI associated with the images so that said 
LOI can be used to control the reproduction of the images (i.e. printing, creating 
thumbnail files for photo albums, etc.)) being arranged to be controlled in response to 
the saliency signal (as discussed in page 6, line 31 - page 7, line 8 , Metcalfe discloses 
that the saliency signal (LOI) can be used to control the reproduction of the images (i.e. 
printing, creating thumbnail files for photo albums, etc.)) (See page 4, line 5 - page 5, 
line 23; page 6, line 23 - page 7, line 8). 
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Metcalfe does not explicitly disclose that the operation in response to the 
saliency signal is in addition to recording the saliency signal in the memory. 

However, Takahashi discloses an electronic camera (See figs. 4(a), 4(b), 8, 9(a), 
9(b), 10, and 11) producing image signal, comprising a user operable picture taking 
control (shooting button 104 as shown in fig. 4 (b)) for selectively activating the 
electronic camera to take pictures, and an additional operable user control (Takahashi 
discloses the use of buttons 109 to set auxiliary information (which the Examiner is 
interpreting as the saliency information) related to the image data being captured; page 
6, If 01 06. Takahashi further discloses the use of a pressure sensor 1 09a and sweat 
sensor 109b to determine the auxiliary information related to the image data (page 6, If 
01 12 - page 7, If 01 14). Also Takahashi discloses that the auxiliary information 
includes information related to the persons (i.e. son, daughter, friend, father, mother) 
and a degree of importance of said persons appearing in the image data (see figs. 23(a) 
and 23 (b); page 8, If 01 17 - page 9, If 0124)) for receiving an input from a user and for 
generating, in response to the input from the user a saliency signal (auxiliary information 
having information such as persons information and degree of importance of the 
persons in the image) that (a) can change in value between at least three different 
discrete values while the image signal is being produced (as shown in figs. 23(a) and 
23(b), the persons information and degree of importance of the persons in the image 
can change in value between a plurality of values (i.e. three or more values as 
claimed)), or (b) can have values that are continuously variable while the image signal is 
being produced (the auxiliary information can be changed during the capture of the 
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image signal), and a memory (13) arranged for storing the image signal and the 
saliency signal (Takahashi further teaches storing the auxiliary information in the header 
of the scene; page 8, If 01 23 - page 9, U 01 26), operation of at least a part of the 
camera apparatus while the electronic camera is activated to take pictures (Takahashi 
discloses recording the auxiliary information and further discloses that the camera 
would also display the auxiliary information on a display screen (As shown in figs. 23, 
the user would set the auxiliary information and the set auxiliary information would be 
displayed in the display as shown in figs. 25 and 26; page 8, U 01 18 - page 9, U 0124)) 
being arranged to be controlled in response to the saliency signal (based on the 
auxiliary information the display would display the auxiliary information as shown in figs. 
25 and 26), the operation in response to the saliency signal is in addition to recording 
the saliency signal in the memory (The Examiner is interpreting displaying the auxiliary 
information using the display of the camera as the operation that is in addition to 
recording the saliency signal in the memory as claimed). Displaying the auxiliary 
information while capturing the image data is advantageous because it would provide 
the user with a user friendly interface that would allow changing the degree of 
importance of the images being captured and would also allow the user to be aware of 
the information being added to the image data. 

Therefore, taking the combined teaching of Metcalfe in view of Takahashi as a 
whole, it would have been obvious to one of an ordinary skill in the art at the time the 
invention was made to apply the concept of having the camera further displaying the 
salient signal being assign to the image data as taught in Takahashi to modify the 
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teaching of Metcalfe to perform an operation in response to the saliency signal in 
addition to recording the saliency signal in the memory. The motivation to do so would 
have been to provide the user with a user friendly interface that would allow changing 
the degree of importance of the images being captured and would also allow the user to 
be aware of the information being added to the image data. 

Regarding claim 3, the combined teaching of Metcalfe in view of Takahashi as 
discussed and analyzed in claim 1 further teaches that wherein said part includes a 
buffer for receiving said image signal, the buffer having a capacity arranged to be 
controlled by the value of the saliency signal during operation of the camera apparatus 
(In a further embodiment, Takahashi discloses the concept of determining the amount of 
data to be transmitted based on the cost of transmission service; wherein the allowed 
length of data (L) is determined and compared to the amount of data to be transmitted, 
if the amount of data to be transmitted the apparatus would select video section with a 
priority higher than a threshold value. If after selecting those video with the priority 
higher than a threshold, the length of data still more than the allowed length, the 
apparatus would increase the priority threshold, however, if the length is less that the 
allowed length, the video signals are transmitted (Page 12, If 0151-0154). By teaching 
adjusting the amount of data to be transmitted based on the length allowed for 
transmission and also based on the priority of the video signals to be transmitted, 
Takahashi inherently discloses "a buffer for receiving said image signal, the buffer 
having a capacity arranged to be controlled by the value of the saliency signal during 
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operation of the camera apparatus" as claimed since the amount being adjusted in the 
apparatus needs to be in a particular buffer or memory prior to transmission in order to 
properly select the video signals with higher priority). One of an ordinary skill in the art 
would have found obvious to apply the concepts of adjusting the amount of data to be 
transmitted based on the priority set to the video signals to further modify the camera 
apparatus to have a buffer for receiving said image signal, the buffer having a capacity 
arranged to be controlled by the value of the saliency signal during operation of the 
camera apparatus with the motivation of select as many as possible the most important 
video signals from the video data for transmission as suggested by Takahashi (Page 
12,H0154). 

Regarding claim 4, the combined teaching of Metcalfe in view of Takahashi as 
discussed and analyzed in claim 1 further teaches that the part includes image selection 
circuitry for receiving the saliency and image signals and for selectively passing ones of 
said image signals as determined by said saliency signal (As taught in Metcalfe, the 
camera receives the saliency signal (LOI) and based on said saliency signal, when 
reproducing, the camera would select particular images based on the degree of 
importance as set by the user when recording the images. By teaching that the images 
with a high LOI would be selected for creating an album or to be printed with high 
quality further teaches that images that do not have a high LOI would be passed or 
skipped since only the images with a high LOI would be displayed in the virtual album. 
Metcalfe as applied reads on "... said part includes image selection circuitry for 
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receiving the saliency and image signals and for selectively passing ones of said image 
signals as determined by said saliency signaf as written since the display or skip of 
images is determined based on the LOI set. (See pages 5-7, specifically page 6, line 31 
- page 7, line 8)). 

Regarding claim 5, the combined teaching of Metcalfe in view of Takahashi as 
discussed and analyzed in claim 1 further teaches that said part comprises the memory 
(memory 13 in Takahashi), the memory including management circuitry arranged to be 
responsive to the saliency signal for selectively retaining in said memory images 
associated with higher saliency levels in preference to images with lower saliency levels 
(Takahashi further discloses that the images are recorded or transmitted based on the 
importance level of the image, wherein only images with high importance level can be 
recorded in order to reduce the amount of use of the recording medium; page 5, U 
0094). 

Regarding claim 10, the combined teaching of Metcalfe in view of Takahashi as 
discussed and analyzed in claim 1 further teaches a user operable control for picture 
taking control of the electronic camera (Metcalfe, 1 1 1 as shown in fig. 1). 

Regarding claim 11, Metcalfe discloses that the user control includes a normal 
picture taking control on the electronic camera (111 as shown in fig. 1). 
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Regarding claim 12, the combined teaching of Metcalfe in view of Takahashi as 
discussed and analyzed in claim 1 further teaches at least one further physically or 
mechanically operable user control for generating a corresponding related saliency 
signal (As shown in Takahashi, figs. 4(b): 109; fig. 9(a): 109, 10: 109a, and 11: 109b, 
Takahashi discloses the use of a plurality of buttons to select from different auxiliary 
information to be assigned to the video signal (note that the buttons 109 are physically 
located on the camera). Furthermore, as shown in figs 23(a) and 23(b), Takahashi 
further discloses that the auxiliary information can be selected using a monitor, wherein 
the user can select the person information and the degree of importance of the persons 
using buttons 103m, 103n, 101m and 101n; page 8, U 0119 (although these buttons are 
displayed on the touch screen display, the Examiner understands that the buttons are 
physically located on the camera since they have different physical location on said 
display)). Grounds for rejecting claim 1 apply here. 

Regarding claim 13, the combined teaching of Metcalfe in view of Takahashi as 
discussed and analyzed in claims 1 and 12 further teaches saliency circuitry for 
combining said saliency signals to form a complex saliency signal (the Examiner is 
reading the complex saliency signal as the combined information having the persons 
information and the degree of importance information as shown in Takahashi), the 
complex saliency signal being the saliency signal for controlling at least a part (The 
Examiner is reading the part as the display of the camera in Takahashi as discussed in 
claim 1) of the electronic camera and the saliency signal the memory is arranged to 
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store (As discussed in claim 1, Takahashi discloses that the auxiliary information has 
persons information and degree of importance of said persons and that the auxiliary 
information is stored with the image signal and further discloses that the camera would 
also display the auxiliary information on a display screen (As shown in figs. 23, the user 
would set the auxiliary information and the set auxiliary information would be displayed 
in the display as shown in figs. 25 and 26 (see persons information and the degree of 
importance displayed on the display); page 8, 1f 0118 - page 9, If 0124)) being arranged 
to be controlled in response to the saliency signal (based on the auxiliary information 
the display would display the auxiliary information as shown in figs. 25 and 26). 
Grounds for rejecting claim 1 and 12 apply here. 

Regarding claim 14, Metcalfe discloses a saliency circuitry for generating an 
image related saliency signal in response to said image signal (Metcalfe discloses 
controlling the reproduction operation of the camera based on the associated saliency 
signal to the image signal so that when reproducing the image signal with higher 
importance would be displayed differently from the other image signal. See page 4, line 
5 - page 5, line 23; page 6, line 23 - page 7, line 8. Takahashi further discloses 
reproducing the video based on the importance level of the image, wherein only images 
with high importance level can be reproduced so that the user can enjoy the recorded 
work without feeling tired, and the power consumption is reduced to secure more driving 
time; page 10, H 01 31). 
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Regarding claim 15, limitations have been discussed and analyzed in claim 13. 

Regarding claim 16, Metcalfe discloses circuitry for incorporating said saliency 
signal in each of said image signals (Metcalfe discloses incorporating said saliency 
signal (LOI) to each of the frames in the image signal. See page 4, line 5 - page 5, line 
23; page 6, line 23 - page 7, line 8). 

Regarding claim 17, the combined teaching of Metcalfe in view of Takahashi as 
discussed and analyzed in claims 1 teaches that the user control is part of the camera 
or is physically attached to the camera body (See Metcalfe, controls in Fig. 1 ; see also 
Takahashi buttons 109 as shown in figs. 4(b), 9(b), 10 and 11, and buttons 103m, 103n, 
101m, 101 n as shown in figs. 23(a) and 23(b)). 

Regarding claim 18, he combined teaching of Metcalfe in view of Takahashi 
fails to teach that the user control is a remote control for communication with the 
camera. However, the Examiner takes Official Notice that the concept of controlling a 
camera with a remote control is well known in the art at the time the invention was made 
and that it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to have made the user control of Metcalfe and Takahashi a remote 
control as opposed to a camera-body integrated control. One would have been 
motivated to do so because it is well known in the art that by using a remote control to 
control some elements of a camera, the user does not have to be near the camera to 
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send and receive desired signals from the camera. This is particularly advantageous in 
cases where plural cameras are used or cameras are placed out of the reach of the 
user (e.g. surveillance cameras), where the remote control would allow the user to send 
signals to the camera(s) from a separate location, thereby simplifying camera control for 
the user. 

Regarding claim 19, Metcalfe discloses that the user control comprises a 
physically movable control member (pressure button that assign a level of interest of an 
image signal based on the pressure applied to said button) and a sensor arranged to be 
responsive to movement of the control member (See page 4, line 27 - page 5, line 4). 
By teaching a pressure button that assigns a LOI of an image signal based on the 
pressure applied to the button, Metcalfe discloses a physically movable control member 
(in this case the pressure button as discussed in Metcalfe) and a sensor arranged to be 
responsive to movement of the control member since by teaching that the LOI is 
assigned based on the pressure applied to the button, the use of a sensor to determine 
the applied pressure in order to assigned the LOI of an image is inherent and 
necessitated in Metcalfe. 

Regarding claim 20, the Examiner notes that the limitations "the user control 
comprises a pressure or force sensing transducer for deriving the saliency signal that 
can have values that are continuously variable", the elements "pressure" or "force 
sensing transducer" are written as optional elements by using the word "or". 
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Metcalfe discloses that the user control comprises a pressure or force sensing 
transducer for deriving the saliency signal that can have values that are continuously 
variable (Metcalfe as applied to claim 19, teaches the use of pressure (pressure button 
that assign a level of interest of an image signal based on the pressure applied to said 
button; page 4, line 27 - page 5, line 4) for deriving the saliency signal. By teaching 
that Metcalfe discloses the use of pressure applied to the button, Metcalfe discloses the 
use of pressure for deriving the saliency signal as claimed. 

Regarding claim 21, limitations have been discussed and analyzed in claim 1. 

Regarding claim 23, limitations have been discussed and analyzed in claim 4. 

Regarding claim 24, limitations have been discussed and analyzed in claim 3. 

Regarding claim 25, limitations have been discussed and analyzed in claim 5. 

Regarding claim 28, limitations have been discussed and analyzed in claim 10. 

Regarding claim 29, limitations have been discussed and analyzed in claim 11. 

Regarding claim 30, limitations have been discussed and analyzed in claim 12. 
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Regarding claim 31, limitations have been discussed and analyzed in claim 13. 

Regarding claim 32, limitations have been discussed and analyzed in claim 13. 

Regarding claim 33, limitations have been discussed and analyzed in claim 13. 

Regarding claim 34, limitations have been discussed and analyzed in claim 16. 

Regarding claim 35, limitations have been discussed and analyzed in claim 17. 

Regarding claim 36, limitations have been discussed and analyzed in claim 18. 

Regarding claim 37, limitations have been discussed and analyzed in claim 19. 

Regarding claim 38, limitations have been discussed and analyzed in claims 19 
and 20. 

Regarding claim 40, claim 40 recites "... one of the saliency signals being a 
signal that (a) can change in value between at least three different discrete values while 
the image signal is being produced, or (b) can have values that are continuously 
variable while the image signal is being produced". It is noted by the Examiner that the 
term " can " is non-limiting and therefore has not been given patentable weight during 
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examination of the claims on their merits. Language that suggests or makes optional 
but does not require steps to be performed or does not limit a claim to a particular 
structure does not limit the scope of a claim or claim limitation. MPEP §21 06. 
The subject matter of a properly construed claim is defined by the terms that limit its 
scope. It is this subject matter that must be examined. As a general matter, the 
grammar and intended meaning of terms used in a claim will dictate whether the 
language limits the claim scope. Language that suggests or makes optional but does 
not require steps to be performed or does not limit a claim to a particular structure does 
not limit the scope of a claim or claim limitation. The following are examples of language 
that may raise a question as to the limiting effect of the language in a claim: 

(A) statements of intended use or field of use, 

(B) "adapted to" or "adapted for" clauses, 

(C) "wherein" clauses, or 

(D) "whereby" clauses. 

This list of examples is not intended to be exhaustive. See also MPEP § 21 1 1 .04. 
USPTO personnel are to give claims their broadest reasonable interpretation in light of 
the supporting disclosure. In re Morris, 127 F.3d 1048, 1054-55, 44 USPQ2d 1023, 
1027-28 (Fed. Cir. 1997). Limitations appearing in the specification but not recited in the 
claim should not be read into the claim. E-Pass Techs., Inc. v. 3Com Corp., 343 F.3d 
1364, 1369, 67 USPQ2d 1947, 1950 (Fed. Cir. 2003) (claims must be interpreted "in 
view of the specification" without importing limitations from the specification into the 
claims unnecessarily). In re Prater, 415 F.2d 1393, 1404-05, 162 USPQ 541, 550- 551 
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(CCPA 1969). See also In re Zletz, 893 F.2d 319, 321-22, 13 USPQ2d 1320, 1322 
(Fed. Cir. 1989) ("During patent examination the pending claims must be interpreted as 
broadly as their terms reasonably allow.... The reason is simply that during patent 
prosecution when claims can be amended, ambiguities should be recognized, scope 
and breadth of language explored, and clarification imposed.... An essential purpose of 
patent examination is to fashion claims that are precise, clear, correct, and 
unambiguous. Only in this way can uncertainties of claim scope be removed, as much 
as possible, during the administrative process."). 

Metcalfe discloses an imaging system (See fig. 1) comprising an electronic 
camera (See fig. 1) for producing an image signal, physically or mechanically operable 
user controls (See user controls 106, 104, 111, 112, 110 and 108 as shown in fig. 1), 
the user control being arranged for receiving an input from a user and for generating 
first saliency signal (Metcalfe discloses the use of button 1 12 to generate a plurality of 
saliency signals (Level of interest signals "LOI") to be associated to the image signal 
being recorded with the camera) while the image signal is being produced, and saliency 
circuitry (the camera in Metcalfe inherently has a saliency signal circuitry to generate 
the saliency signal upon operation of the camera button 1 1 2) for storing said first 
saliency signal (Metcalfe discloses recording the plurality of saliency signals in a 
memory (tape 120 in fig. 1) in association with the image data; see page 4, line 5 - 
page 5, line 23; page 6, line 23 - page 7, line 8), the saliency signal being, a signal that 
(a) can change in value between at least three different discrete values while the image 
signal is being produced (Metcalfe discloses assigning a level of interest to the image 
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data being recorded, wherein the user can variably assign a plurality of level of interest 
through the capture of the a video sequence; see Fig. 3. This teaches generating a 
saliency signal that can change in value between at least three different discrete values 
while the image signal is being produced), or (b) can have values that are continuously 
variable while the image signal is being produced (Metcalfe discloses assigning a level 
of interest to the image data being recorded, wherein the user can variably assign a 
plurality of level of interest through the capture of the a video sequence; see Fig. 3. 
This teaches generating a saliency signal that can have values that are continuously 
variable while the image signal is being produced), a memory (120 as shown in fig. 1) 
arranged for storing the image signal and the saliency signal in response to the saliency 
signal (page 4, lines 5-13; page 5, lines 4-35), operation of at least part of the electronic 
camera being arranged to be controlled in response to the saliency signal (as discussed 
in page 6, line 31 - page 7, line 8, Metcalfe discloses that the saliency signal (LOI signal 
stored in memory 120) can be used to control the reproduction of the images (i.e. 
printing, creating thumbnail files for photo albums, etc.)) (See page 4, line 5 - page 5, 
line 23; page 6, line 23 - page 7, line 8). 

Metcalfe does not explicitly disclose at least two physically or mechanically 
operable user controls, each of the user controls for generating first and second 
saliency signals; that said saliency circuitry combines said first and second saliency 
signals to form a complex saliency signal; that the operation of at least part of the 
electronic camera being arranged to be controlled in response to the complex saliency 
signal. 
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However, Takahashi discloses an electronic camera (See figs. 4(a), 4(b), 8, 9(a), 
9(b), 10, and 11) producing image signal, comprising a user operable picture taking 
control (shooting button 104) for selectively activating the electronic camera to take 
pictures, and at least two physically or mechanically operable user controls (As shown 
in Takahashi, figs. 4(b): 109; fig. 9(a): 109, 10: 109a, and 11: 109b, Takahashi discloses 
the use of a plurality of buttons to select from different auxiliary information (which the 
Examiner is interpreting as the saliency information) to be assigned to the video signal 
(note that the buttons 109 are physically located on the camera). Furthermore, as 
shown in figs 23(a) and 23(b), Takahashi further discloses that the auxiliary information 
can be selected using a monitor, wherein the user can select the person information 
and the degree of importance of the persons using buttons 103m, 103n, 101m and 
1 01 n; page 8, U 01 1 9 (although these buttons are displayed on the touch screen 
display, the Examiner understands that the buttons are physically located on the 
camera since they have different physical location on said display)). Takahashi further 
discloses the use of a pressure sensor 109a and sweat sensor 109b to determine the 
auxiliary information related to the image data (page 6, U 01 1 2 - page 7, If 01 1 4). Also 
Takahashi discloses that the auxiliary information includes information related to the 
persons (i.e. son, daughter, friend, father, mother) and a degree of importance of said 
persons appearing in the image data (see figs. 23(a) and 23 (b); page 8,^0117- page 
9, 1| 0124)) for receiving an input from a user and for generating, in response to the 
input from the user a saliency signal (auxiliary information having information such as 
persons information and degree of importance of the persons in the image) that (a) can 



Application/Control Number: 10/696,567 Page 37 

Art Unit: 2622 

change in value between at least three different discrete values while the image signal 
is being produced (as shown in figs. 23(a) and 23(b), the persons information and 
degree of importance of the persons in the image can change in value between a 
plurality of values (i.e. three or more values as claimed)), or (b) can have values that are 
continuously variable while the image signal is being produced (the auxiliary information 
can be changed during the capture of the image signal), and saliency circuitry for 
combining said saliency signals to form a complex saliency signal (the Examiner is 
reading the complex saliency signal as the combined information having the persons 
information and the degree of importance information as shown in Takahashi), the 
complex saliency signal being the saliency signal for controlling at least a part (The 
Examiner is reading the part as the display of the camera in Takahashi as discussed in 
claim 1) of the electronic camera and the saliency signal the memory is arranged to 
store (As discussed in claim 1, Takahashi discloses that the auxiliary information has 
persons information and degree of importance of said persons and that the auxiliary 
information is stored with the image signal and further discloses that the camera would 
also display the auxiliary information on a display screen (As shown in figs. 23, the user 
would set the auxiliary information and the set auxiliary information would be displayed 
in the display as shown in figs. 25 and 26 (see persons information and the degree of 
importance displayed on the display which the examiner is reading as the complex 
saliency signal generated) (This teaches performing an operation ); page 8, If 01 18 - 
page 9, U 0124)) being arranged to be controlled in response to the saliency signal 
(based on the auxiliary information the display would display the auxiliary information as 
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shown in figs. 25 and 26). Therefore Takahashi discloses at least two physically or 
mechanically operable user controls, each of the user controls for generating first and 
second saliency signals; that said saliency circuitry combines said first and second 
saliency signals to form a complex saliency signal; that the operation of at least part of 
the electronic camera being arranged to be controlled in response to the complex 
saliency signal as claimed. Having a plurality of operation controls to generate different 
saliency signals to be combined into a single complex saliency signal to control the 
display operation of the camera is advantageous because it would provide the user with 
a user friendly interface that would allow changing the degree of importance of the 
images being captured and would also allow the user to be aware of the information 
being added to the image data. 

Therefore, taking the combined teaching of Metcalfe in view of Takahashi as a 
whole, it would have been obvious to one of an ordinary skill in the art at the time the 
invention was made to apply the concept of having the camera with a plurality of 
operation controls to generate different saliency signals to be combined to control the 
operation of the display to further displaying the combined saliency signal being assign 
to the image data as taught in Takahashi to modify the teaching of Metcalfe to have at 
least two physically or mechanically operable user controls, each of the user controls for 
generating first and second saliency signals; that said saliency circuitry combines said 
first and second saliency signals to form a complex saliency signal; that the operation of 
at least part of the electronic camera being arranged to be controlled in response to the 
complex saliency signal. The motivation to do so would have been to provide the user 
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with a user friendly interface that would allow changing the degree of importance of the 
images being captured and would also allow the user to be aware of the information 
being added to the image data. 

Regarding claim 42, Metcalfe discloses a separate user operable picture taking 
control for selectively activating the electronic camera to take pictures (See button 1 1 1 
as shown in fig. 1). 

Regarding claim 54, claim 54 recites "an electronic camera having a picture 
taking control for selectively activating the camera to derive input picture signals, the 
electronic camera further including a user operable control for generating a saliency 
signal capable of having plural values and a buffer for receiving the input picture signals 
and having a capacity for the input picture signals determined in response to the value 
of the saliency signal". It is noted by the Examiner that the term " capable of . It is noted 
by the Examiner that the term "capable of is non-limiting and therefore has not been 
given patentable weight during examination of the claims on their merits. Language that 
suggests or makes optional but does not require steps to be performed or does not limit 
a claim to a particular structure does not limit the scope of a claim or claim limitation. 
MPEP §2106. 

The subject matter of a properly construed claim is defined by the terms that limit 
its scope. It is this subject matter that must be examined. As a general matter, the 
grammar and intended meaning of terms used in a claim will dictate whether the 
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language limits the claim scope. Language that suggests or makes optional but does 
not require steps to be performed or does not limit a claim to a particular structure does 
not limit the scope of a claim or claim limitation. The following are examples of language 
that may raise a question as to the limiting effect of the language in a claim: 

(A) statements of intended use or field of use, 

(B) "adapted to" or "adapted for" clauses, 

(C) "wherein" clauses, or 

(D) "whereby" clauses. 

This list of examples is not intended to be exhaustive. See also MPEP § 21 1 1 .04. 

USPTO personnel are to give claims their broadest reasonable interpretation in 
light of the supporting disclosure. In re Morris, 127 F.3d 1048, 1054-55, 44 USPQ2d 
1023, 1027-28 (Fed. Cir. 1997). Limitations appearing in the specification but not recited 
in the claim should not be read into the claim. E-Pass Techs., Inc. v. 3Com Corp., 343 
F.3d 1364, 1369, 67 USPQ2d 1947, 1950 (Fed. Cir. 2003) (claims must be interpreted 
"in view of the specification" without importing limitations from the specification into the 
claims unnecessarily). In re Prater, 415 F.2d 1393, 1404-05, 162 USPQ 541, 550- 551 
(CCPA 1969). See also In re Zletz, 893 F.2d 319, 321-22, 13 USPQ2d 1320, 1322 
(Fed. Cir. 1989) ("During patent examination the pending claims must be interpreted as 
broadly as their terms reasonably allow.... The reason is simply that during patent 
prosecution when claims can be amended, ambiguities should be recognized, scope 
and breadth of language explored, and clarification imposed.... An essential purpose of 
patent examination is to fashion claims that are precise, clear, correct, and 
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unambiguous. Only in this way can uncertainties of claim scope be removed, as much 
as possible, during the administrative process"). 

Furthermore, it has been held that the recitation that an element is "capable of 
performing a function is not a positive limitation but only requires the ability to so 
perform. It does not constitute a limitation in any patentable sense. In re Hutchison, 69 
USPQ 138. 

Metcalfe discloses an apparatus (See fig. 1) comprising an electronic camera 
(See fig. 1 ) having a picture taking control (button 1 11 as shown in fig. 1 ) for selectively 
activating the camera to store an image to a memory (Fig. 1 : 120), the camera further 
including a user operable control (button 1 12 as shown in fig. 1 ) for generating a non- 
playback saliency signal (the saliency signal generated in Metcalfe is a level of interest 
signal to indicate portion of the image signals that have certain degree of interest to be 
stored in the memory 120 in association with the saliency signal) and picture selection 
circuitry (circuitry is inherent in the Metcalfe reference to control the recording and 
reproduction of the video signals stored in the memory 120) for selectively passing the 
picture signals in response to the saliency signal (Metcalfe discloses that he user can 
select a saliency signal (LOI) to a particular image being captured, the saliency signal is 
stored in association with the image (See pages 5-7). Metcalfe further discloses that 
based on the stored saliency signal, the images would be reproduced, wherein when 
reproducing, the camera would select particular images based on the degree of 
importance (LOI) as set by the user when recording the images (See pages 5-7; page 6, 
line 31 - page 7, line 8)). By teaching that the images with a high LOI would be 
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selected for creating an album or to be printed with high quality further teaches that 
images that do not have a high LOI would be passed or skipped since only the images 
with a high LOI would be displayed in the virtual album. Metcalfe as applied reads on 
the limitations ". . . picture selection circuitry for selectively passing the picture signals in 
response to the saliency signal ..." since the display or skip of images is determined 
based on the saliency signal (LOI)), the saliency signal being capable of having more 
than two values (As discussed by Metcalfe , the operation of the button 112 would 
automatically generate a plurality of saliency signals, wherein when operating the button 
112, if the button is set to be controlled by pressing it a plurality of times, it would 
represent creating a first plurality of saliency signals that would generate the plurality of 
saliency signals to be associated to the image signal when recording into the memory 
120) (See page 4, line 5 - page 5, line 23; page 6, line 23 - page 7, line 8). 

Although Metcalfe discloses selectively passing the image signals during 
reproduction of the video based on the saliency signal, Metcalfe does not explicitly 
disclose selectively passing the image to the memory in response to the saliency signal. 

However, Takahashi discloses an electronic camera (See figs. 4(a), 4(b), 8, 9(a), 
9(b), 10, and 11) producing image signal, comprising a user operable picture taking 
control (shooting button 104 as shown in fig. 4 (b)) for selectively activating the 
electronic camera to take pictures for further recording or transmission (Takahashi 
discloses that the captured image data would be either recorded or transmitted to 
another device (See page 5, U 0094-0097)), and an additional operable user control 
(Takahashi discloses the use of buttons 109 to set auxiliary information (which the 
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Examiner is interpreting as the saliency information) related to the image data being 
captured; page 6, U 0106. Takahashi further discloses the use of a pressure sensor 
109a and sweat sensor 109b to determine the auxiliary information related to the image 
data (page 6, ^ 0112 - page 7, If 01 14). Also Takahashi discloses that the auxiliary 
information includes information related to the persons (i.e. son, daughter, friend, father, 
mother) and a degree of importance of said persons appearing in the image data (see 
figs. 23(a) and 23 (b); page 8, U 01 17 - page 9, H 0124)) for receiving an input from a 
user and for generating, in response to the input from the user a saliency signal 
(auxiliary information having information such as persons information and degree of 
importance of the persons in the image) that (a) can change in value between at least 
three different discrete values while the image signal is being produced (as shown in 
figs. 23(a) and 23(b), the persons information and degree of importance of the persons 
in the image can change in value between a plurality of values (i.e. three or more values 
as claimed)), or (b) can have values that are continuously variable while the image 
signal is being produced (the auxiliary information can be changed during the capture of 
the image signal), operation of at least a part of the camera apparatus while the 
electronic camera is activated to take pictures (Takahashi discloses recording the 
auxiliary information and further discloses that the camera would also display the 
auxiliary information on a display screen (As shown in figs. 23, the user would set the 
auxiliary information and the set auxiliary information would be displayed in the display 
as shown in figs. 25 and 26; page 8, If 01 1 8 - page 9, If 01 24)) being arranged to be 
controlled in response to the saliency signal (based on the auxiliary information the 
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display would display the auxiliary information as shown in figs. 25 and 26). Also, 
Takahashi teaches that the important images from a captured video can be extracted so 
that only scenes of high degrees of importance can be efficiently recorded or 
transmitted as compared with the case where all of the shot scenes are stored or 
transmitted and that the extraction of the scenes of high degrees of importance may be 
carried out during or after shooting of the scenes (See page 5, If 0094; page 10, If 0130) 
(This teaches the concept of selectively passing the image to the memory in response 
to the saliency signal as claimed since the image signals would be either recorded or 
transmitted based in the priority given to the scenes so that only important scenes are 
transmitted or recorded while said scenes are being captured). 

Therefore, taking the combined teaching of Metcalfe in view of Takahashi as a 
whole, it would have been obvious to one of an ordinary skill in the art at the time the 
invention was made to apply the concept of selectively recording image data in a 
memory in response to an assigned priority to scenes in the image data as taught in 
Takahashi to modify the teaching of Metcalfe to selectively pass the image to the 
memory in response to the saliency signal. The motivation to do so would have been to 
reduce the amount of use in the recording medium and reduce the communication costs 
as suggested by Takahashi (Page 5, If 0094). 



Regarding claim 56, limitations have been discussed and analyzed in claim 54. 
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9. Claims 2, 7, and 22 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Metcalfe, AU 743216 B in view of Takahashi, US 2002/0041757 
A1 and further in view of Matsumoto et al., US Patent 6,795,642 B2. 

Regarding claim 2, although Takahashi teaches that a compression circuitry 
(See Takahashi, fig. 1:15; page 5, 1f 0097; page 6, U 0105) for receiving the image 
signals and for compressing the image signals and that although the invention is 
described on the premise that a shot picture is recorded, a shot picture is not 
necessarily recorded, and it can be used also when compressed video and audio data 
are transmitted as they are to be used on a network or the like (page 10, U 0130), the 
combined teaching of Metcalfe in view of Takahashi fails to teach compressing the 
image signals to an extent determined by the saliency signal. 

However, Matsumoto et al. teaches the concept of having a video recording 
apparatus (Fig. 2) of a surveillance system, recording video data captured by an 
electronic camera (See fig. 1), wherein when an alarm is activated, the importance of 
the video is determined to be high as compared to when the alarm is not activated (Col. 
3, lines 46-67). Matsumoto et al. further discloses that based on the degree of 
importance given to the video signal, the data compression is also adjusted (i.e. if the 
importance degree of the video is low, it would be compressed at high level and if the 
importance degree of the video is high, said video would be compressed at low level) 
(Col. 3, line 1 5 - col. 4, lines 1 9). Matsumoto also discloses that the importance level 
can also be adjusted by the user operating the surveillance system (Col. 7, lines 22-34). 
Matsumoto et al. further discloses that by adjusting the compression of the video being 
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captured, it is possible to record the monitoring image data having a high degree of 
importance as much as possible (Col. 7, lines 35-43). 

Therefore, taking the combined teaching of Metcalfe in view of Takahashi and 
further in view of Matsumoto et al. as a whole, it would have been obvious to one of 
ordinary skill in the art a the time the invention was made to apply the concept of 
adjusting the compression of a video captured by an electronic camera based on a 
degree of importance assign to the video as discussed in Matsumoto et al. to modify the 
teaching of Metcalfe and Takahashi by compressing the image signals to an extent 
determined by the saliency signal. The motivation to do so would have been to record 
the monitoring image data having a high degree of importance as much as possible as 
suggested in Matsumoto et al. 

Regarding claim 7, the combined teaching of Metcalfe in view of Takahashi as 
discussed and analyzed in claims 1 and 2 further teaches that said part comprises the 
memory (memory 13 in Takahashi), including management circuitry arranged to be 
responsive to the saliency signal for selectively retaining in said memory images 
associated with higher saliency levels in preference to images with lower saliency levels 
(Takahashi further discloses that the images are recorded or transmitted based on the 
importance level of the image, wherein only images with high importance level can be 
recorded in order to reduce the amount of use of the recording medium; page 5, U 
0094). 
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Regarding claim 22, limitations have been discussed and analyzed in claim 2. 

10. Claims 51-53 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Takahashi, US 2002/0041757 A1. 

Regarding claim 51, claim 51 recites "an electronic camera having a picture 
taking control for selectively activating the camera to derive input picture signals, the 
electronic camera further including a user operable control for generating a saliency 
signal capable of having plural values and a buffer for receiving the input picture signals 
and having a capacity for the input picture signals determined in response to the value 
of the saliency signal". It is noted by the Examiner that the term " capable of . It is noted 
by the Examiner that the term "capable of is non-limiting and therefore has not been 
given patentable weight during examination of the claims on their merits. Language that 
suggests or makes optional but does not require steps to be performed or does not limit 
a claim to a particular structure does not limit the scope of a claim or claim limitation. 
MPEP §2106. 

The subject matter of a properly construed claim is defined by the terms that limit 
its scope. It is this subject matter that must be examined. As a general matter, the 
grammar and intended meaning of terms used in a claim will dictate whether the 
language limits the claim scope. Language that suggests or makes optional but does 
not require steps to be performed or does not limit a claim to a particular structure does 
not limit the scope of a claim or claim limitation. The following are examples of language 
that may raise a question as to the limiting effect of the language in a claim: 
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(A) statements of intended use or field of use, 

(B) "adapted to" or "adapted for" clauses, 

(C) "wherein" clauses, or 

(D) "whereby" clauses. 

This list of examples is not intended to be exhaustive. See also MPEP § 21 1 1 .04. 

USPTO personnel are to give claims their broadest reasonable interpretation in 
light of the supporting disclosure. In re Morris, 127 F.3d 1048, 1054-55, 44 USPQ2d 
1023, 1027-28 (Fed. Cir. 1997). Limitations appearing in the specification but not recited 
in the claim should not be read into the claim. E-Pass Techs., Inc. v. 3Com Corp., 343 
F.3d 1364, 1369, 67 USPQ2d 1947, 1950 (Fed. Cir. 2003) (claims must be interpreted 
"in view of the specification" without importing limitations from the specification into the 
claims unnecessarily). In re Prater, 415 F.2d 1393, 1404-05, 162 USPQ 541, 550- 551 
(CCPA 1969). See also In re Zletz, 893 F.2d 319, 321-22, 13 USPQ2d 1320, 1322 
(Fed. Cir. 1989) ("During patent examination the pending claims must be interpreted as 
broadly as their terms reasonably allow.... The reason is simply that during patent 
prosecution when claims can be amended, ambiguities should be recognized, scope 
and breadth of language explored, and clarification imposed.... An essential purpose of 
patent examination is to fashion claims that are precise, clear, correct, and 
unambiguous. Only in this way can uncertainties of claim scope be removed, as much 
as possible, during the administrative process.") 

Furthermore, it has been held that the recitation that an element is "capable of 
performing a function is not a positive limitation but only requires the ability to so 
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perform. It does not constitute a limitation in any patentable sense. In re Hutchison, 69 
USPQ 138. 

In a first embodiment, Takahashi discloses an apparatus comprising an 
electronic camera (See figs. 4(a), 4(b), 8, 9(a), 9(b), 10, and 11) having a picture taking 
control (shooting button 104 as shown in fig. 4 (b)) for selectively activating the camera 
to derive input picture signals, the electronic camera further including a user operable 
control for generating a saliency signal (Takahashi discloses the use of buttons 109 to 
set auxiliary information (which the Examiner is interpreting as the saliency information) 
related to the image data being captured; page 6, U 0106. Takahashi further discloses 
the use of a pressure sensor 109a and sweat sensor 109b to determine the auxiliary 
information related to the image data (page 6, 1f 01 1 2 - page 7, If 011 4). Also 
Takahashi discloses that the auxiliary information includes information related to the 
persons (i.e. son, daughter, friend, father, mother) and a degree of importance of said 
persons appearing in the image data (see figs. 23(a) and 23 (b); page 8, U 01 1 7 - page 
9, 1| 0124)) capable of having plural values (as shown in figs. 23(a) and 23(b), the 
persons information and degree of importance of the persons in the image can change 
in value between a plurality of values). 

In said first embodiment, Takahashi does not explicitly disclose a buffer for 
receiving the input picture signals and having a capacity for the input picture signals 
determined in response to the value of the saliency signal. 

However, in a further embodiment Takahashi discloses the concept of 
determining the amount of data to be transmitted based on the cost of transmission 
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service; wherein the allowed length of data (L) is determined and compared to the 
amount of data to be transmitted, if the amount of data to be transmitted the apparatus 
would select video section with a priority higher than a threshold value. If after selecting 
those video with the priority higher than a threshold, the length of data still more than 
the allowed length, the apparatus would increase the priority threshold, however, if the 
length is less that the allowed length, the video signals are transmitted (Page 12, If 
0151-0154). By teaching adjusting the amount of data to be transmitted based on the 
length allowed for transmission and also based on the priority of the video signals to be 
transmitted, Takahashi inherently discloses "a buffer for receiving said image signal, the 
buffer having a capacity arranged to be controlled by the value of the saliency signal 
during operation of the camera apparatus" as claimed since the amount being adjusted 
in the apparatus needs to be in a particular buffer or memory prior to transmission in 
order to properly select the video signals with higher priority). Takahashi further 
discloses that by adjusting the amount of data to be transmitted based on the length 
allowed for transmission and also based on the priority of the video signals to be 
transmitted the camera may select as many as possible the most important video 
signals from the video data for transmission (Page 12, If 0154). 

Therefore, taking the combined teaching of the first and the further embodiments 
in Takahashi, one of an ordinary skill in the art would have found obvious at the time the 
invention was made to apply the concepts of adjusting the amount of data to be 
transmitted based on the priority set to the video signals to further modify the camera 
apparatus to have a buffer for receiving said image signal, the buffer having a capacity 



Application/Control Number: 10/696,567 Page 51 

Art Unit: 2622 

arranged to be controlled by the value of the saliency signal during operation of the 
camera apparatus with the motivation of select as many as possible the most important 
video signals from the video data for transmission as suggested by Takahashi (Page 
12,110154). 

Regarding claim 52, limitations have been discussed and analyzed in claim 51 . 
Regarding claim 53, limitations have been discussed and analyzed in claim 51 . 
Conclusion 

1 1 . THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the mailing date of this final action. 
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Contact 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to NELSON D. HERNANDEZ whose telephone number is 
(571 )272-731 1 . The examiner can normally be reached on 9:00 A.M. to 5:30 P.M. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Lin Ye can be reached on (571) 272-7372. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 
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